Protocol for coordinating the distribution of shared memory

ABSTRACT

Methods, systems, and articles of manufacture consistent with the present invention coordinate distribution of shared memory to threads of control executing in a program by using a cooperative synchronization protocol. The protocol serializes access to memory by competing threads requesting assignment of memory space, while allowing competing threads that have already been assigned memory space, to share access to the memory. A designated area of memory assigns the memory to requesting threads. The protocol is an application level entity and therefore does access the operating system to serialize the memory allocation process.

FIELD OF THE INVENTION

This invention relates generally to multi-threaded applications, andmore particularly to methods for coordinating the distribution of sharedmemory to threads of control executing in a data processing system.

BACKGROUND OF THE INVENTION

Multi-threading is the partitioning of a computer program that performsa particular process into logically independent “threads” of controlthat can execute in parallel. Each thread includes a sequence ofinstructions and data used by the instructions to carry out a particularprogram task, such as a computation or input/output function.

Although a single processor can execute instructions of only one threadat a time, the processor can execute multiple threads in parallel by,for example, executing instructions corresponding to one thread untilreaching a selected instruction, suspending execution of that thread,and executing instructions corresponding to another thread, until allthreads have completed. In this scheme, as long as the processor hasbegun executing instructions for more than one thread during a giventime interval all executing threads are said to be “running” during thattime interval, and multi-processing is achieved.

A process includes multiple threads operating out of a common addressspace. Processes may be configured such that portions of their addressspace are shared, that is, are mapped to a common physical location, ora memory. Multiple threads within a process share portions of thatmemory address space. It is possible for more than one thread to believeit has sole access to a portion of memory and for it to perform a seriesof reads and writes to an area of memory while another thread isperforming reads and writes to the same area of memory. Such concurrentoperation by competing threads may confuse operation of a process.Therefore, in a shared memory environment the shared memory must beallocated to each thread based on a coordination technique ensuring thatonly one thread has access to an area of shared memory at a time.

Current memory allocation techniques assign address space to threads inone of two ways: distribution or mutual exclusion. Distributiontechniques partition shared memory into equal-sized blocks and assignthe memory blocks to threads as requested. Each memory block must belarge enough to meet the memory requirements of the thread requiring themost memory. Total memory requirements are dictated by the maximumdemand for access to shared memory of a given thread multiplied by thetotal number of threads involved. This method is particularly wastefulwhen individual threads do not require similar amounts of shared memory.Even slight differences in the threads' memory requirements can have asignificant impact on the total memory requirement.

Mutual exclusion techniques, which may be implemented in hardware orsoftware, coordinate use of shared memory among threads by reading andwriting shared variables and pointers referring to the shared addresses.Implementation of these techniques requires accessing the operatingsystem, which is both slow and expensive.

In a mutual exclusion implementation the operating system only permitsone thread at a time to access shared memory. Shared memory is assignedto a thread by exchanging signals between threads, such that a threadcan be forced to stop at a specified point during execution until it hasreceived a specific signal. More specifically, the operating system mayallow a thread to access shared memory by locking the shared memory,allocating the memory for exclusive use of the thread, and unlocking thememory once the thread has completed the memory operation.

While the memory area is locked other threads are prevented fromaccessing it until the lock has been removed. This method is anacceptable way of allocating memory for one-time or infrequentoperations, for example, the one-time startup required by a process.However, computing generally requires execution of operations that occurhundreds or even thousands of times per second, for example, when manythreads are modifying a common memory area by performing a “write”operation. For such operations this method of locking and unlockingshared memory creates bottlenecks in system performance. As the numberof threads making demands on system performance increases, thelikelihood of a system bottleneck increases. This of course negativelyimpacts system performance.

SUMMARY OF THE INVENTION

Methods, systems, and articles of manufacture consistent with thepresent invention allocate blocks of a shared memory to threads duringexecution in a data processing system. Methods, systems, and articles ofmanufacture consistent with the present invention, as embodied andbroadly described herein, involve operations that include receiving amemory allocation request, determining whether access to the sharedmemory is permitted for purposes of the allocation request, identifyinga next available memory block in response to the memory allocationrequest, and designating the next available memory block in response tothe request.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this specification, illustrate an implementation of theinvention and, together with the description, serve to explain theadvantages and principles of the invention. In the drawings,

FIG. 1 depicts a block diagram of a data processing system suitable forpracticing methods and systems consistent with the present theinvention;

FIG. 2 depicts a logical view of multiple threads accessing a sharedmemory;

FIG. 3 depicts a block diagram used to explain the operation of a sharedmemory system operating in accordance with a shared memory protocolconsistent with the present invention;

FIG. 4 depicts a physical representation of threads being allocatedblocks of a shared memory in a manner consistent with the presentinvention; and

FIG. 5 depicts a flow chart illustrating operations performed by ashared memory allocation and coordination process consistent with animplementation of the present invention.

DETAILED DESCRIPTION

Reference will now be made in detail to an implementation consistentwith the present invention as illustrated in the accompanying drawings.Wherever possible, the same reference numbers will be used throughoutthe drawings and the following description to refer to the same or likeparts.

Overview

In accordance with methods and systems consistent with the presentinvention, a programmer writing a multi-threaded application programspecifies in the program a total amount of memory space to be shared bythe program threads during execution. The programmer also specifies a“block size,” that is, the size of each partition of the total memoryspace to be allocated on an as-needed basis to the threads duringexecution. Blocks of the shared memory space are allocated to thethreads serially, meaning that when a thread requires memory (forexample, the computational task being performed by the thread exceedsthe thread's current memory allocation), the thread determines whetherits request can be processed and, if so, the thread is allocated thenext available block in the shared memory space. Each thread may have aunique identifier and all thread identifiers may be provided at thebeginning of program execution. Memory blocks may be assigned to thethreads using the identifiers. Information may also be maintained in adesignated area of the shared memory to identify each thread by itsidentifier as well as the specific block(s) of shared memory assigned toeach thread.

A next available block indicator located in a designated area of theshared memory maintains information identifying the location (i.e.,beginning address) in the shared memory for the next available block.Since all blocks are the same size, the indicator is simply updated by afixed value based on the block size at the conclusion of each memoryallocation cycle.

The programmer also specifies a shared memory protocol for the threads.This protocol, which may be included as part of the program itself orseparate from the program, consists of a set of rules governing the wayin which threads cooperate to access the shared memory for purposes ofperforming the memory allocation function. According to this protocol,when a thread requires memory, it first determines whether anotherthread is currently accessing the shared memory space for memoryallocation purposes. This may be done by providing a “token” in adesignated area of the shared memory space. The needy thread determineswhether the token is available, meaning that no other thread isaccessing the shared memory space to be allocated a memory block. If thetoken is available, the needy thread “takes” the token (i.e., indicatesthat the token is not currently available) and proceeds to be allocatedthe next available block.

If, however, the token is “taken,” meaning that another thread isaccessing the shared memory space to be allocated a memory block, theneedy thread does not proceed. Rather, the needy thread monitors thetoken, waiting for it to become available, at which time the thread canbe allocated a next available memory block.

Once a thread completes the allocation process, it releases the token,thus making the token available for other memory allocation cycles. Atthe completion of an allocation cycle the next available block indicatoris updated so that the next time a thread seeks to be allocated a memoryblock the indicator reflects the next available block.

Although access to the shared memory space is exclusive to a singlethread for purposes of the allocation process, all of the threadsoperating in parallel can access their designated block(s), even whenthe allocation process for a particular thread is ongoing. The onlyrestriction in this scheme relates to memory block allocation, only onethread at a time can be allocated a block. Additionally, all of thethreads potentially have access to all of the shared memory space. Inother words, no thread has exclusive control over a particular block orset of blocks specified at the beginning of program execution.

Computer Architecture

FIG. 1 depicts an exemplary data processing system 100 suitable forpracticing methods and implementing systems consistent with the presentinvention. Data processing system 100 includes a computer system 105connected to a network 190, such as a Local Area Network, Wide AreaNetwork, or the Internet.

Computer system 105 contains a main memory 125, a secondary storagedevice 130, a processor 140, an input device 150, and a video display160. All of these components operate in a manner commonly understood bythose skilled in the art. For example, processor 140 executes anoperating system and application programs. Input device 150 receivesuser input and video display 160 displays output, for example, from anapplication program being executed by processor 140.

Main memory 125 and secondary storage device 130 may be used to storeall of portions of the operating system and applications programs whenthey are not being executed by processor 140. For purposes of thisdescription, however, main memory 125 is shown in FIG. 1 as includingboth an operating system 120 and application program 110. Operatingsystem 120 represents any operating system, although an operating systemthat specifically facilitates multi-threaded program execution may bepreferred, for example, the Solaris® operating system from SunMicrosystems, Inc.

Application program 110 is a multi-threaded program that performs aparticular process using independent, logical threads of control thatexecute in parallel on processor 140. Although a single processor isshown, one skilled in the art will appreciate multiple processors may beused to implement methods and practice systems consistent with thepresent invention.

During execution of application program 110, its threads share access tomain memory 125 and secondary storage 130. Specific implementations ofmemory in a data processing system may use a virtual memory to storesome of the data associated with a process, allowing main memory tostore data associated with more processes. In that type of systemthreads also share access to the virtual memory area.

Application program 110 includes a shared memory protocol 115. Sharedmemory protocol 115 provides a set of guidelines for threads to followwhen accessing memory. Shared memory protocol 115 interacts with thethreads of an application running in program 110 and with memory 125.One skilled in the art will appreciate that other implementationsconsistent with the present invention may implement shared memoryprotocol 115 separate from program 110 in either software or hardware.

Although aspects of this implementation are depicted as being stored inmemory 125, one skilled in the art will appreciate that all or part ofsystem and methods consistent with the present invention may be storedon or read form other computer-readable media, such as secondary storagedevices, like hard disks, floppy disks, and CD-ROM; a digital signalreceived from a network such as the Internet; or other forms of ROM orRAM.

FIG. 2 depicts a logical representation of multiple threads sharingaccess to memory. When multiple threads share access to memory, each ofthe threads is able to access any portion of the memory. FIG. 2 showsmultiple threads 210 _(a)-210 _(n) sharing access to memory 200. Becauseeach of the threads 210 _(a)-210 _(n) may simultaneously access all ofmemory, it is possible for more than one thread use a portion of memorybeing used by another thread, confusing an operation. Therefore, threadsoperating in a shared memory environment compete for use of memory.Methods and systems consistent with the present invention are directedto coordinating the distribution of access to shared memory by competingthreads.

Shared-Memory Allocation and Distribution System

FIG. 3 is a logical representation of a shared memory configurationconsistent with the present invention. Threads 304, 306, 308, and 310may correspond to one or more processes that are being executedconcurrently by processor 320. Again, although one processor 320 isshown in FIG. 3, methods and systems consistent with the presentinvention may operate in multi-processor environments as well.

Memory 340 includes a file allocation table (“FAT”) 345. To facilitateparallel execution of multiple threads, memory 340 is logicallypartitioned into segments of equal size, as designated by a user of ashared memory system operating in accordance with the principles of theshared memory protocol 330. The partition size is used to assign a blockof memory, corresponding to address space, to a thread.

When an application program begins execution and spawns its threads,each thread agrees to access memory 340 according to the rules of sharedmemory protocol 330. Shared memory protocol 330 describes the format ofthe FAT 345 to each thread. According to protocol 330, each threadagrees to view an indicator indicating whether another thread isaccessing memory, to be allocated a block, before securing memory foritself. The protocol 330 allows threads that have already secured accessto a block of memory to use that block for the duration of theapplication's execution. Each time a thread needs to secure access to anew block of memory, it must follow the rules of the shared memoryprotocol 330.

For example, if thread 304 has secured access to memory block 350, itmay access that block of memory for the duration of a program'sexecution. However, while thread 306 secures access to memory block 355,thread 304 may not attempt to secure access to another block of memory340. While thread 306 secures access to a block of memory 340, thread304 may only access the block(s) of memory it has already secured. Oncethe indicator in the FAT 345 indicates that thread 306 has completed itssecure operation, thread 304 may attempt to secure an additional blockof memory.

The FAT 345 is used to assign a block of memory to a thread. Included inthe FAT 345 are tables corresponding to current assignments of blocks ofmemory to threads using shared memory 340, an indicator reflectingwhether a thread is currently attempting to secure access to a block ofmemory, and a pointer indicating a next available memory block. A threadsecures access to a block of memory by receiving an assignment of ablock of memory from the FAT. When a thread secures a block of memory,it increments a pointer in the FAT to the next available memory block,shown in FIG. 4.

FIG. 4 is a physical representation of how a shared memory 400 may beallocated to threads and maintained by methods and systems consistentwith the present invention. Before a program executes, a user specifiesboth the size of a shared memory 400 and the size of individual memoryblocks 415, 420, 425, and 430. Each memory block corresponds to a set ofaddresses. Once the size of the shared memory 400 and the size of itsblocks 415, 420, 425, and 430 have been assigned, they remain constantthroughout a program's execution.

Each thread that accesses memory 400 in accordance with the principlesof the present invention is uniquely identifiable. Upon a first threadin a process beginning to execute, the process is assigned a processidentification code. When a thread receives assignment of a memory blockit is assigned a thread identification code and a memory location code.Each thread of a program may be uniquely identified by a global threadidentification code, a combination of a process identification code forthe process corresponding to the thread, and a thread identificationcode. This identification feature enables methods and systems consistentwith the present invention to handle any parallel processingapplication, including multi-threaded applications as well asmulti-processor applications running processes containing multiplethreads.

Further details on operation of a shared memory system operating inaccordance with the principles of the shared memory protocol will now beexplained with reference to the flow diagram of FIG. 5. Shared memoryprotocol begins with user initialization. During initialization, theuser's program specifications are set for the size of both the sharedmemory and of the individual memory blocks to be assigned to threadsaccessing memory (step 505). The size of the shared memory area and thesize of the memory blocks do not affect the operation of a shared memoryallocation or distribution process consistent with the presentinvention. Therefore, the user may specify the sizes by using any numberof criteria, for example, the memory requirements of a majority of thethreads running.

Before attempting to secure access to a block of memory a threaddetermines whether another thread is securing access to a block ofmemory. If a thread determines that a “token” to secure access to memoryis available (step 530), the thread secures a block of memory for itselfby accessing the FAT (step 535).

Upon securing access to a block of memory, the FAT is used to assign thethread a memory location code representing the memory location of thememory block and a thread identification code, used to identify thethread within a process (step 535). Because threads are automaticallyassigned a next available memory block, threads of different processesmay be assigned consecutive memory blocks. Each thread receives a memoryblock corresponding to the size of the memory blocks set by the userduring initialization 505. If a thread requires more memory than thatcontained in an individual memory block, the thread will be assignedenough memory blocks to meet its request.

Once all of the location and identification assignments have been made,a memory assignment counter, maintained in the FAT region of the sharedmemory, is incremented (step 540). Incrementing the counter initiatessetting a pointer to a next available memory location. Once a thread hassecured access to a block of memory, the thread releases the “token” sothat another thread may secure access to a block of memory (step 545).

A thread may place as many memory requests as needed. A shared memorysystem operating in accordance with the principles of the shared memoryprotocol ensures that all executing threads needing memory have anopportunity to secure access to memory by repeating the memoryassignment process until all threads have completed execution (step550).

CONCLUSION

Methods and systems consistent with the present invention coordinatedistribution of shared memory among threads of control executing in aparallel computing environment. Distributing shared memory resources ina manner consistent with the principles of the present invention doesnot require accessing the operating system, a costly and time consumingevent, to distribute shared memory among threads. Nor does a systemconsistent with the principles of the present invention requireprohibiting competing threads from accessing a shared memory area whileanother thread accesses the memory.

Systems consistent with the present invention are applicable to allmulti-threaded programs written in all computer programming languages,including Fortran 77, Java, C, and C++.

Although the foregoing description has been described with reference toa specific implementation, those skilled in the art will know of variouschanges in form and detail which may be made without departing from thespirit and scope of the present invention as defined in the appendedclaims and the full scope of their equivalents.

What is claimed is:
 1. A method for allocating blocks of a shared memoryin a data processing system, the system including a first processcontaining a first thread, and a second process containing a secondthread, wherein the first and second process are assigned a first andsecond process identification code, respectively, the method comprising:accessing by the first thread a file allocation table in the sharedmemory to secure access to a first block of the shared memory based onthe determination that a token to secure memory access is available tothe first thread; securing the token by the first thread based on thedetermination; assigning, by the file allocation table, a first memorylocation code to the first thread, wherein the memory location codecorresponds to a memory location of the first block; incrementing by thefirst thread a counter maintained in the file allocation table, whereinthe counter points to an available memory block within the sharedmemory; releasing the token by the first thread; securing the token bythe second thread; and assigning, by the file allocation table, a secondmemory location code to the second thread, wherein the second memorylocation code corresponds to a memory location of the available memoryblock within the shared memory, and wherein assigning the second memorylocation code takes place while the first thread accesses the firstblock of the shared memory.
 2. The method of claim 1, wherein assigninga memory location code includes: determining that the first threadrequires memory larger then the size of the first block; and assigningadditional consecutive memory blocks of the shared memory, starting atthe first block, such that the aggregated size of the additional memoryblocks, including the first block, is equal to or greater than the sizeof the required memory for the first thread.
 3. The method of claim 1,wherein assigning a second memory location code to the second threadincludes: incrementing by the second thread the counter maintained inthe file allocation table; and releasing the token by the second thread.4. The method of claim 1, further including: selecting by a user thesize of the blocks of a shared memory, wherein the selection isperformed prior to the first thread accessing the file allocation tableto secure access to the first block of the shared memory.
 5. A methodfor allocating blocks of a shared memory in a data processing system,the method comprising: receiving a memory allocation request from afirst thread; allocating a first block of the shared memory to the firstthread; accessing the first block by the first thread; denying a secondmemory allocation request by the first thread based on the determinationthat a token to secure memory access is secured by a second thread;deallocating the first block from the first thread when the first threadhas completed its use of the first block; releasing the token by thesecond thread; receiving a third memory allocation request from thefirst thread; and allocating a second block of the shared memory to thefirst thread based on the determination that the token is available tothe first thread, wherein the second block is allocated to the firstthread while the second thread is accessing the shared memory.
 6. Amethod for allocating blocks of a shared memory in a data processingsystem, the method comprising: receiving a memory allocation requestfrom a first thread; determining whether a token to secure a memoryblock within the shared memory is available for purposes of the memoryallocation request; identifying an available memory block in the sharedmemory in response to the memory allocation request based on adetermination that access to the shared memory is permitted for purposesof the memory allocation request; securing the token by the firstthread; incrementing a counter that designates another available memoryblock in the shared memory; releasing the token by the first thread;accessing by the first thread the available memory block; receiving asecond memory allocation request from a second thread; securing thetoken by the second thread; allocating the another available memoryblock to the second thread; incrementing the counter; releasing thetoken by the second thread; and accessing by the second thread theanother available memory block.
 7. The method of claim 6, whereindesignating the available memory block in response to the requestincludes: updating an area of memory with an address of an allocatedmemory block.
 8. The method of claim 6, wherein a first process executesthe first thread and a second process executes the at least otherthread.
 9. A memory allocating system, comprising: a shared memory; andat least one processor executing at least one thread, wherein the atleast one processor receives a memory allocation request from a firstthread of the at least one thread, determines whether a token thatenables a thread to secure access to a memory block is available,identifies a next available memory block in response to the memoryallocation request based on a determination that the token is available,allows the first thread to secure the token, allocates the nextavailable memory block to the first thread in response to the request,increments a counter that designates another available memory block inthe shared memory, enables the first thread to release the token oncethe next available memory block is secured, grants access to the nextavailable block, receives a second memory allocation request from asecond thread, allows the second thread to secure the token, allocatesthe another available memory block to the second thread, increments thecounter, enables the second thread to release the token and allows thesecond thread to access the another available memory block.
 10. Thesystem of claim 9, wherein the at least one processor updates an area ofmemory with an address of an allocated memory block.
 11. The system ofclaim 9, wherein a first process executes the first thread and a secondprocess executes the at least one other thread.
 12. A computer-readablemedium containing computer-readable instructions for performing a methodon a data processing system for allocating blocks of a shared memory,the method comprising: receiving a memory allocation request from afirst thread; determining whether a token to secure a memory blockwithin the shared memory is available for purposes of the memoryallocation request; identifying an available memory block in the sharedmemory in response to the memory allocation request based on adetermination that access to the shared memory is permitted for purposesof the memory allocation request; securing the token by the firstthread; incrementing a counter that designates another available memoryblock in the shared memory; releasing the token by the first thread;accessing by the first thread the available memory block; receiving asecond memory allocation request from a second thread; securing thetoken by the second thread; allocating the another available memoryblock to the second thread; incrementing the counter; releasing thetoken by the second thread; and accessing by the second thread theanother available memory block.
 13. The computer-readable medium ofclaim 12, wherein designating the available memory block in response tothe request includes: updating an area of memory with an address of anallocated memory block.
 14. The computer-readable medium of claim 12,wherein a first process executes the first thread and a second processexecutes the at least one other thread.